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SUMMARY 


The  Intelligent  Knowledge-based  Instructional  System  (IKIS)  is  IBM 
PC  based  software  designed  to  teach  operator  skills  for  a  piece  of 
complex  electronic  test  equipment.  This  report  summarizes 
preliminary  research  and  development  which  utilizes  simulation  as 
an  important  component  of  the  learning  process.  Simulating  real 
world  stimulus  situations  in  an  adaptive  learning  environment 
provides  a  mechanism  for  teaching  cognitive,  situational,  and 
procedural  components  that  semantic  knowledge  alone  can  not 
provide. 

Phase  I  of  the  IKIS  project  has  produced  the  basic  architecture 
required  for  further  research  and  system  development  of  adaptive 
simulation  based  training  software.  The  Teledyne  Transponder  Test 
Set,  a  device  used  to  test  aircraft  transponders  on  the  flight 
line,  served  as  the  subject  domain  during  Phase  I.  All  major 
system  components  have  been  created  providing  a  test  bed 
environment  for  adding  deeper  levels  of  knowledge  and  increased 
adaptability  during  later  development.  The  following  report 
describes  each  of  these  components  in  detail  and  relates  this 
research  with  further  research  and  development  being  proposed  for 
the  Phase  II  effort. 

Long  term  goals  of  the  IKIS  project  focus  on  the  development  of 
simulation  based  adaptive  tutoring  to  improve  cognitive  and 
behavioral  learner  performance  of  real-time  tasks.  The  modular 
design  of  the  Phase  I  system  architecture  provides  for  the  ability 
to  add  and  modify  system  components  rapidly  in  an  iterative 
fashion.  Because  of  this  further  research  and  development.  Phase 
II  can  build  on  Phase  I  results  but  is  not  limited  by  the  Phase  I 
environment. 

Methods  for  teaching  knowledge  which  integrates  many  cognitive  and 
behavioral  components  is  a  subject  which  still  requires  theoretical 
research.  Developing  intelligent  tutoring  systems  that  adapt  to 
individual  learners  and  can  model  the  learning  process  effectively 
is  a  related  area  also  of  some  theoretical  research.  Combining 
research  and  development  in  these  related  areas  will  be  essential 
for  cost  eftective  and  efficient  operator  training  on  the 
increasingly  sophisticated  equipment  currently  being  developed  for 
use  in  both  military  and  industrial  applications.  This  research 
will  have  major  impacts  in  training  throughout  the  public  and 
private  sector.  Hence,  the  IKIS  project  provides  a  developmental 
tool  that  can  be  useful  in  advancing  research  in  these  areas,  as 
well  as  a  product  that  can  have  immediate  use  for  the  Army  and  the 
corporate  sponsor. 
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PREFACE 


The  research  described  in  this  report  is  authorized  under  contract 
number  MDA903-87-C-0585  issued  by  Department  of  the  Army  Defense 
Supply  Service-Washington. 

The  report  describes  worked  completed  for  Phase  I  of  a  three  phase 
U.S.  Department  of  Defense  SBIR  research  project.  It  relates  the 
work  completed  with  work  to  be  done  in  Phase  II  and  Phase  III  of 
this  project. 

MICROEXPERT  Systems  is  greatly  indebted  to  the  Teledyne  Electronics 
Company  and  especially  Mr.  John  Taylor.  Without  the  generous  help 
and  information  they  provided  this  research  would  not  have  been 
possible . 
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INTRODUCTION 


IKIS ,  an  Intelligent  Knowledge-based  Instructional  System,  is  a 
project  designed  to  provide  research  and  development  for  an 
intelligent  tutoring  system.  The  IKIS  project  seeks  to  encode  the 
expertise  of  a  teacher  and  technician  combined  with  high  fidelity 
simulation  to  improve  cognitive  and  behavioral  performance  of  the 
learner.  The  following  report  describes  the  results  of  Phase  I 
research  and  development  of  IKIS  and  its  relationship  to  future 
Phase  II  and  Phase  III  efforts. 


PROJECT  GOALS 


The  Phase  I  contract  has  two  major  goals  related  to  the  development 
of  an  Intelligent  Tutoring  System.  They  are  : 

*  A  complete  systems  design  and  specification 

*  A  demonstration  program  to  prove  feasibility 

Within  the  framework  of  the  project  several  subgoals  were 
identified: 

*  Design  a  "friendly"  graphics  oriented  user 
interface . 

*  Develop  a  test  bed  environment  for  investigation  of 
student  behavioral  measurements 

*  Investigate  the  components  needed  for  further 
development  of  an  Intelligent  Tutor  Authoring 
System. 

*  Utilize  a  high  resolution  simulation 

*  Design  for  field  portable  and  embedded  training 


PROJECT  PROCEDURAL  PLAN 


The  Phase  I  project  plan  emphasized  rapidly  prototyping  the  major 
modules  needed  in  an  Intelligent  Tutor.  This  would  provide  an 
iterative  test  bed  environment  for  adding,  testing,  and  revising 
heuristics  and  metaknowledge  during  Phase  II. 


PROJECT  OVERVIEW 


SUBJECT  DOMAIN  SEARCH 


Early  project  design  decisions  suggested  developing  a  tutoring 
system  that  would  adapt  itself  easily  to  a  variety  of  subject 
matter.  This  would  create  an  authoring  tool  whose  usefulness  goes 
beyond  the  initial  project  goals.  When  completed,  IKIS  will  be  an 
authoring  system  that  allows  instructional  experts  to  develop  a 
variety  of  instructional  applications  easily. 

In  choosing  the  subject  domain,  the  main  criteria  included: 

*  Benefits  for  the  corporate  sponsor  as  well  as  Army  goals 

*  Subject  matter  applicable  to  high  fidelity  simulation 

*  Access  to  domain  expertise 

*  Required  effort  commensurate  with  Phase  I  funding 

After  consultation  with  several  potential  corporate  sponsors,  it 
was  determined  that  for  Phase  I  operator  training  on  the  Teledyne 
AN/APM-424 (v) 2  Transponder  Test  would  serve  as  the  subject  domain. 
The  Transponder  Test  Set  is  a  hand  held  device  that  the  Army  has 
recently  chosen  to  purchase  for  testing  aircraft  IFF 
(Identification  Friend  or  Foe)  transponders  on  the  flight  line. 
The  cognitive  learning  tasks  related  to  overall  operation  of  this 
device  are  important  and  r.on-trivial  in  size  and  content.  Training 
of  operators  in  the  field  is  currently  limited  to  an  instruction 
manual.  There  is  no  team  or  classroom  teaching. 

Operation  of  the  Test  Set  requires  the  operator  to  integrate 
procedural  and  factual  relationships  of  test  set  and  transponder 
operation  with  transponder  and  test  set  fault  identification,  plus 
deeper  knowledge  related  to  IFF  theory.  Operators  who  have 

attempted  to  learn  from  only  the  manual  have  reported  frustration 
in  understanding  the  functions  and  relationships  of  the  component 
parts.  According  to  Teledyne  instructors,  field  technicians  are 
likely  not  to  read  the  instruction  manual  but  instead  to  attempt  to 
teach  themselves  by  trial  and  error  experimentation  with  the  test 
set.  It  is  likely  that  this  type  of  learning  will  result  in  the 
incomplete  and  inaccurate  formation  of  important  cognitive 
relationships .  Furthermore  it  could  result  in  damage  to  the  test 
set  and/or  reduce  readiness  if  working  aircraft  are  diagnosed  as 
faulty  due  to  incorrect  test  set  operation. 
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The  subject  domain  lends  itself  well  to  a  high  fidelity  simulation 
as  part  of  the  instructional  design.  Both  Army  field  technicians 
and  the  corporate  sponsor  benefit  from  development  of  the  product. 
The  long  range  development  of  an  authoring  system  for  designing 
simulation  based  intelligent  tutoring  systems  will  have  broad 
applications  for  training  in  both  the  public  and  private  sectors. 

HARDWARE  TRADEOFF 


A  subgoal  of  the  project  was  to  make  IKIS  field  portable.  Teledyne 
Electronics  utilizes  primarily  IBM  compatible  microcomputers.  Thus 
the  designer  chose  an  IBM  compatible  microcomputer  to  be  the 
developmental  system.  The  minimum  configuration  required  to 
fulfill  the  project  goals  and  objectives  include: 

*  Minimum  640  Kilobytes  RAM  memory 

*  20  megabyte  hard  disk 

*  EGA  color  graphics  capability 

In  addition,  some  additional  features  are  desirable: 

*  80286  or  80386  microprocessor 

*  4  megabyte  RAM 


SOFTWARE  TRADEOFF 


A  major  objective  of  Phase  I  was  to  develop  a  rapid  prototype 
designed  to  demonstrate  proof  of  concept  and  serve  as  a  test  bed 
environment  for  iterative  testing  of  teaching  strategies, 
simulation,  heuristical  inferencing  and  the  user  interface. 

The  goals  required  a  language  with: 

*  Modular  code 

*  High  resolution  color  graphics  capability 

*  Mouse  and  windowing  routines  built  in 

*  High  level  for  rapid  prototyping 

A  number  of  expert  systems  shells,  including  M.l  (Teknowledge)  and 
Rulemaster  (Radian),  were  found  to  be  limited  in  their  graphics 
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capability  and  built  in  heuristical  support.  Implementations  of 
LISP  on  the  PC  also  were  rejected,  as  they  lack  strong  graphics 
and  windowing  routines,  required  longer  prototyping  times,  and 
lacked  mouse  input  routines.  For  similar  reasons  languages  such 
as  C  and  Pascal  were  ruled  out. 

The  language  chosen  to  fit  both  software  and  hardware  requirements 
of  Phase  I  was  Smalltalk  V,  recently  released  by  Digitalk  Inc.  It 
provides  EGA  color  graphics  capabilities,  windowing  and  mouse 
routines.lt  is  object  oriented  and  produces  modular  code.  These 
capabilities  suggested  the  ability  to  develop  a  rapid  prototype  of 
the  proposed  system  design.  Once  the  original  design  is  complete, 
changes  and  additions  can  easily  be  made  in  order  to  test  out  new 
ideas  and  refinements  to  the  project.  Furthermore  the  modular 
nature  of  this  object  oriented  language  provides  an  incremental 
method  of  recoding  to  improve  system  execution  during  later  phases 
of  the  project. 


SYSTEM  OVERVIEW 


For  the  Phase  I  SBIR  contract  MICROEXPERT  Systems,  Inc. ,  created 
IKIS ,  an  Intelligent  Knowledge-Based  Instructional  System,  for 
operator  field  training  on "the  Teledyne  AN/APM-424 (v) 2  Transponder 
Test  Set. 

The  Test  Set  is  hand  held  electronic  test  equipment  used  by  field 
technicians  to  test  aircraft  transponders  on  the  flight  line.  The 
Test  Set  is  housed  in  a  metal  box  approximately  1.5  cubic  feet  in 
volume  with  a  battery  pack  on  the  bottom,  a  handle  with  test 
buttons  on  the  side,  and  a  flat  antenna  on  top.  It  has  a 
viewfinder  through  which  the  technician  can  aim  at  an  aircraft's 
antenna  and  read  test  results  on  a  display. 

IKIS  demonstrates  the  major  components  of  a  simulation  based 
intelligent  tutoring  systems  and  serves  as  a  test  bed  for  further 
development  in  this  field.  The  IKIS  Phase  I  prototype  utilizes 
high  fidelity  graphics  to  simulate  the  operation  of  the  Transponder 
Test  Set,  along  with  a  Socratic  style  tutor  and  coach  to  assist  the 
learning  process.  It  demonstrates  the  major  components  required  of 
an  intelligent  tutoring  system  including  an  inference  engine  and 
knowledge  base,  a  student  model,  a  tutoring  strategy,  and  a 
friendly  user  interface. 

The  user  interacts  with  IKIS  primarily  through  the  use  of  a  mouse 
pointing  device  with  some  input  through  the  keyboard.  Interactive 
components  of  IKIS  are  shown  in  different  windows  on  the  computer 
display.  Four  major  window  are  available  to  the  user:  the  Test 
Set,  Tutor,  Viewfinder,  and  Test  Zones  windows. 


The  Test  Set  window,  shown  in  Figure  2,  gives  the  user  an  outside 
view  of  the  Test  Set  and  provides  pop-up  menus  for  simulating  the 
connections  the  user  makes  between  the  Test  Set,  KIR-1  computer  and 
power  supply.  It  further  simulates  the  steps  the  user  must  go 
through  when  loading  secret  codes  into  the  computer  for  use  by  the 
Test  Set. 
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AN/APtt-424  Transponder  Test  Set 

> 

Figure  2 .  Transponder  test  set  window 


Simulation  of  the  Test  Set  includes  a  view  through  the  Test  Set 
Viewfinder.  The  viewfinder  window  is  shown  in  Figure  3. 

Basic  operation  of  the  Test  Set  involves  looking  through  the 
Viewfinder,  aiming  at  an  aircraft  transponder  antenna,  pressing  the 
test  buttons  on  the  outside  handle,  and  reading  the  viewfinder 
display.  Aiming  of  the  Viewfinder  is  simulated  using  the  mouse  to 
position  the  cursor  in  the  desired  directional  arrow,  located  in 
the  upper  right  corner  of  the  window,  and  pressing  the  left  mouse 
button.  The  outside  Test  Set  switches  (buttons)  are  shown  again 
through  a  port  hole  at  the  upper  left  of  the  screen.  Using  the 
mouse  the  user  can  simulate  pressing  the  switches  to  perform  the 
various  tests  reguired.  LED's  and  lamps  are  simulated  to  display 
the  readout  that  would  result  from  the  test  performed. 
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Passive  coaching  is  provided  if  the  learner  simply  points  at  a 
viewfinder  component  and  clicks  the  left  mouse  button.  A  pop-up 
displays  appears  providing  information  about  that  component  to  the 
learner.  Figure  3  illustrates  coaching  related  to  the  Accept  Lamp 
that  was  requested  by  the  user. 


Kioua  Lou  Zone 
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Test  Set  Switches 
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Accept  Lamp-  -  indicates  test  passed 


Figure  3.  Viewfinder  window 


Correct  aiming  of  the  test  set  also  requires  the  user  to  be 
positioned  in  a  preferred  test  zone.  If  the  user  is  outside  of  a 
test  zone,  the  transponder  antenna  will  be  shielded  from 
interrogation  by  the  test  set.  A  map  of  Test  zones  corresponding 
to  each  aircraft  type  is  shown  in  a  separate  window.  An  example  of 
the  lower  test  zone  of  the  Kiowa  helicopter  is  shown  in  figure  4. 
The  position  of  the  user  is  simulated  by  a  dot  on  this  map.  The 
learner  can  reposition  himself  on  the  map  by  pointing,  with  the 
mouse,  to  an  area  on  the  map  and  clicking  the  left  mouse  button. 

The  socratic  tutor  is  provided  in  a  separate  window  for  the 
learner.  The  tutor  provides  semantic  knowledge  related  to 
operation  of  the  Test  Set.  From  the  tutor  the  learner  is  free  to 
activate  any  other  window  in  order  to  simulate  the  operations  being 
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KNOWLEDGE  ENGINEERING 


INTRODUCTION 


A  variety  of  types  of  knowledge  can  be  entered  into  an  Intelligent 
Tutor.  The  exact  nature  of  the  knowledge  required  to  optimize 
individual  learning  will  require  additional  research  and  was  not 
the  goal  of  Phase  I.  What  was  desired  during  Phase  I  was  to  design 
the  basic  components  required  for  entering  heuristical  knowledge 
into  the  system. 


SOURCES  OF  KNOWLEDGE 


A  complete  intelligent  tutoring  system  requires  several  sources  and 
types  of  knowledge.  The  expertise  of  a  human  expert  knowledgeable 
in  the  subject  matter  is  most  obvious.  A  second  source  of 
knowledge  for  tutoring  is  the  knowledge  used  by  an  expert  human 
teacher.  Frequently  such  experts  have  gained  their  skills  from  a 
combination  of  education  and  years  of  practical  experience.  Often 
such  experts  cannot  articulate  the  path  they  took  to  their 
conclusions  and  decisions.  They  chunk  information  into  much  larger 
cognitive  packages  then  neophytes.  Their  reasoning  process  has 
had  such  subtle  stimulus  strengthening  that  it  appears  to  be 
intuitive. 

The  main  sources  of  knowledge  for  the  Phase  I  effort  comprise  the 
APM  424  student  training  manual,  the  A  PM  424  technical  manual,  and 
Mr.  John  Taylor,  an  Instructor  for  Teledyne  Electronics  Inc.  who  is 
experienced  with  the  transponder  test  set.  The  information  gleaned 
from  these  sources  was  redesigned  and  modified  for  inclusion  in  the 
IKIS  project  by  MICROEXPERTS ’  instructional  psychologist. 

If  training  of  field  operation  of  the  Transponder  Test  Set  is  to 
remain  the  subject  domain  of  the  Phase  II  effort,  actual  flight 
line  functioning  of  the  device  and  an  experienced  operator  will  be 
required.  Furthermore  a  review  of  the  operation  of  the  simulation 
should  be  carried  out  by  a  technician  who  is  completely  familiar 
with  the  functioning  and  potential  faults  of  the  test  set. 
Finally,  student  testing  and  revision  will  be  required  to  produce 
the  final  product. 
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CHARACTERIZATION  OF  PROBLEM  DOMAIN 


As  described  briefly  above,  the  problem  domain  consisted  primarily 
of  operational  training  and  fault  identification  at  the  field 
level.  Operation  of  the  Transponder  Test  set  on  the  flight  line 
involves  correctly  performing  each  of  the  following  procedures: 

*  Connection  of  the  Test  Set  to  the  KIR  computer  cable 

*  Connection  to  an  appropriate  power  supply 

*  Connection  to  KIR-1  computer 

*  Loading  of  Mode  4  code  into  KIR-1 

*  Loading  of  Mode  4  into  Test  Set 

*  Recognizing  the  name  and  function  of  all  components 

*  Performing  the  lamp  test 

*  Correct  aiming  of  test  set  at  an  IFF  antenna 

*  Stepping  through  the  Test  Sequence 

*  Discriminating  operator  faults 

*  Performing  the  test  repeat  function 


CONCEPTS  &  RELATIONS  FOR  PROBLEM  SOLVING 


An  aircraft  transponder  is  designed  to  respond  to  a  "query"  by 
sending  back  electronic  signals  indicating  that  it  is  friendly.  If 
the  transponder  does  not  work  correctly,  the  aircraft  could  easily 
be  interpreted  as  a  foe.  A  properly  functioning  transponder  is 
essential  to  the  well-being  and  effectiveness  of  the  aircraft  and 
its  pilot.  The  Transponder  Test  Set  is  designed  to  detect  faults 
in  the  aircraft's  transponder.  If  the  Test  Set  has  been  operated 
correctly  and  is  functioning  properly,  then  a  defective  aircraft 
transponder  is  indicated  by  the  readout  seen  in  the  viewfinder 
display. 

The  field  operator  of  the  Transponder  test  Set  must  learn  to 
discriminate  between  three  types  of  problems  related  to  functioning 
of  the  Test  Set.  Errors  can  result  from: 

*  Faults  in  the  Transponder 


*  Faults  in  the  Test  Set 

*  Incorrect  Test  Set  operation 

It  is  imperative  that  the  Test  Set  operator  be  familiar  with  Test 
Set  operation,  the  Test  Set  display  and  the  meaning  of  each  test 
lamp  and  LED  readout  that  can  occur.  This  knowledge  is  important 
for  proper  Test  Set  operation  and  fault  identification. 

The  Test  Set  fault  codes  are  displayed  in  the  lower  right-hand 
corner  LED  of  the  viewfinder.  Codes  displayed  are  numbered  0 
through  8,  F,  and  A.  Each  code  corresponds  to  a  probable  fault, 
usually  in  the  transponder.  Code  F  indicates  a  failure  in  the  Test 
Set  itself  while  code  A  indicates  an  angle  deflection  indicating 
incorrect  operator  position.  The  learner  can  review  the  meaning  of 
each  code  from  the  tutor. 

Faults  in  the  Test  Set  are,  in  most  cases,  indicated  by  the  BIT 
(built-in-test)  that  is  performed  prior  to  every  mode  test.  As 
explained  above,  code  "F"  in  the  lower  right-hand  corner  indicates 
a  Test  Set  fault.  This  assumes  that  the  code  LED  is  itself 
operational.  The  operator  must  be  sure  to  perform  the  lamp  test 
prior  to  using  the  Test  Set  and  at  any  time  that  a  lamp  or  LED 
failure  could  produce  an  anomalous  readout.  The  simulator 

simulates  the  functioning  of  the  lamp  test  by  allowing  the  user  to 
click  on  the  lamp  test  button  using  a  mouse  pointing  device. 

Faults  are  most  likely  to  occur  due  to  incorrect  operation  of  the 
test  set.  Faults  can  result  from  performing  any  of  the  following 
operational  procedures  incorrectly: 

*  Aiming  of  the  test  set 

*  Positioning  in  preferred  test  zone 

*  Disconnection  of  KIR-1  computer  and  cables 

*  Activating  test  sequence  and  test  repeat  buttons 

*  Loading  of  mode  4  code  into  test  set 

The  ability  of  the  operator  to  identify  these  faults  and  effect  the 
appropriate  repair  procedure  is  crucial  to  his  function. 

To  test  the  operator's  ability  to  diagnose  faults  a  fault  can  be 
inserted  into  the  Test  Set  or  transponder  from  the  top  viewfinder 
menu.  The  user  is  made  aware  that  a  fault  is  present  but  the  exact 
nature  of  the  fault  must  be  discovered.  The  learner  is  free  to 
consult  the  tutor  during  this  process.  When  identified,  the  fault 
can  be  repaired  by  mousing  on  the  correct  repair  procedure  from  the 
fault  repair  menu. 
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MODELING  STUDENT  PERFORMANCE 


No  intelligent  communication  can  take  place  without  understanding 
the  learner.  Information  about  the  learner  and  the  learner's 
behavior  is  stored  in  the  student  model.  The  adaptability  of  an 
intelligent  tutoring  system  is  largely  determined  by  the 
qualitative  and  quantitative  student  behavioral  measurements 
contained  and  accessed  in  the  model. 

Observable  behavior,  in  many  cases,  can  have  multiple  sources  of 
strengthening.  The  cognitive  components  that  evoke  a  given 
behavior  can  only  be  inferred.  A  major  goal  of  the  Phase  I 
research  was  to  specify  the  basic  components  of  the  student  model 
carrying  out  detailed  Phase  II  research. 

As  currently  implemented,  the  student  model  stores  student 
performance  history  including  the  current  simulation  state,  plus 
faults  that  have  not  been  correctly  diagnosed  by  the  learner. 
These  facts  are  added  to  the  knowledge  base  during  program 
execution.  In  this  way  the  inference  engine  has  access  to  facts 
related  to  the  current  student  so  that  didactic  decisions  can  be 
made  that  adapt  to  that  student. 

Most  importantly,  the  student  model  provides  a  test  bed  for  further 
research  into  relationships  between  observable  behavioral  events 
and  inferred  student  cognitions.  The  student  model,  as 
implemented,  is  an  object  with  its  data  and  variables  hidden  from 
other  objects  in  the  systems.  Additional  behavioral  measurements 
can  easily  be  inserted  along  with  additions  and  modifications  in 
the  inference  rules.  Performance  measurements  can  then  be  used  to 
determine  the  appropriateness  of  the  inferencing  being  carried  out. 

KNOWLEDGE  BASE 


The  knowledge  base  consists  of  facts  together  with  rules  that 
operate  on  those  facts.  The  heuristics  gleaned  from  a  human  expert 
or  experts  are  captured  in  the  knowledge  base.  As  mentioned  above, 
the  human  expertise  required  for  an  intelligent  tutoring  system 
includes  both  the  expertise  of  one  or  more  subject  domain  experts 
as  well  as  the  heuristical  knowledge  of  an  expert  teacher.  The 
implementation  of  this  knowledge  is  still  theoretical  as  well  as 
empirical.  The  communications  band  between  the  user  and  the  expert 
system  interface  is  obviously  narrower  and  quite  different  from  the 
rich  communication  between  a  student  and  human  tutor.  The  task  of 
constructing  a  model  of  the  learner  or  the  expert  is  not  a  simple 
one  for  computer  based  systems.  IKIS  provides  a  test  bed 
environment  for  empirical  research  into  the  use  of  simulations 
combined  with  heuristics  that  optimize  learner  performance. 


KNOWLEDGE  REPRESENTATION 


Knowledge  can  be  represented  hierarchically  as  zeroth  order,  first 
order,  and  second  order  knowledge. 

Zeroth  order  knowledge  consists  of  the  basic  facts,  procedures,  and 
relationships  about  the  subject  domain.  It  is  low  level  "textbook" 
knowledge.  IKIS  provides  this  to  the  user  through  the  socratic 
style  tutor,  built  in  coaching  mechanisms,  and  context  sensitive 
help  screens. 

The  heuristical  knowledge  embodying  the  rules  of  thumb  and 
judgmental  criteria  of  an  expert  is  sometimes  classified  as  the 
first  order  correction  to  the  zeroth  order  factual  theory  and 
begins  to  lend  some  intelligence  to  the  system.  IKIS  employs 
heuristical  knowledge  related  to  the  faults  that  may  occur,  user 
induced  or  otherwise,  in  operation  of  the  transponder  test  set. 

A  second  order  correction  to  the  domain  knowledge  is  metaknowledge, 
knowledge  about  knowledge.  Such  knowledge,  when  incorporated  into 
the  system,  can  provide  auxiliary  information  to  the  inference 
engine.  This  information  can  be  used  to  provide  deeper  answers  to 
questions,  restrict  the  sea’rch  space  for  a  solution,  and  control  a 
more  adaptive  user  interface.  Implementation  of  metaknowledge  is 
beyond  the  scope  of  the  Phase  I  research.  However  the  modular 
nature  of  the  inference  engine  provides  an  easy  mechanism  for 
creating  whole  new  instances  of  the  rule  base  if  desired.  Research 
to  be  carried  out  in  Phase  II  will  focus  on  identifying  and 
implementing  this  level  of  knowledge. 


TASK  ANALYSIS 


The  designers  performed  a  task  analysis  to  identify  the  nature  and 
quantity  of  knowledge  that  can  be  captured  in  the  system.  This 
involves  identification  of  appropriate  behaviors  and  behavioral 
sequences  along  with  the  knowledge  required  to  correctly  operate 
the  Transponder  Test  Set. 

TERMINAL  OBJECTIVES 


Army  training  objectives  related  to  the  Transponder  Test  Set 
requires  that,  without  the  aid  of  notes,  using  only  supplied 
operators  manual,  the  student  will  be  able  to 
perform/describe,  orally  or  in  writing,  the  following: 

*  Purpose  of  the  AN/APN-424 ( v)  2  Transponder  Test 
Set. 

*  Unpacking/Preparation  for  use. 

*  Operational  Characteristics  of  the  TS-4077/APM- 
424 (v)  2. 

*  Analyze  Display  of  the  TS-4077/APM-424  (v)2. 

*  Testing  IFF  Systems  with  the  TS-4077/APM-424  (v)2. 

*  Operator  Corrective  Action/Unit  Level  Maintenance 
Steps . 

IKIS  designers  identified  a  subset  of  these  objectives  that 
were  rewritten  in  the  form  of  behavioral  and  cognitive 
objectives. 


BEHAVIORAL  OBJECTIVES 


The  Phase  I  project  concentrated  on  operation  and  fault  diagnosis 
of  the  Test  Set.  Unpacking  and  preliminary  preparation  procedures 
were  not  included.  Following  completion  of  IKIS  the  student  should 
be  able  to: 

*  Recognize  and  describe  the  names  and  functions  of  all 
test  set  switches,  viewfinder  LED's  and  lamps. 


*  Correctly  recognize  c  1  interpret  any  and  all 
possible  viewfinder  displays. 

*  Correctly  aim  the  test  set  at  aircraft  IFF  antenna. 

*  Identify  and  list  the  correct  steps  required  to  load 
mode  4  code  into  the  test  set. 

*  Using  the  simulator,  perform  the  lamp  test  and  test 
set  verification  procedures. 

*  Connect  and  disconnect  KIR-l  computer,  cables,  test 
set,  and  power  supply,  using  the  simulator. 

*  Recognize  and  indicate  the  correct  repair  or 
replacement  procedure  when  a  fault  occurs  in  the  test 
set,  transponder,  KIR-1  computer,  or  cables. 


COGNITIVE  OBJECTIVES 


Beyond  teaching  of  rote  behavioral  responses,  the  system  should 
teach  a  higher  level  of  understanding  of  the  procedural  and  factual 
relationships  between  component  parts  of  the  learning  tasks. 
Cognitive  theory  suggests  that  knowledge  is  first  acquired 
declaratively  through  instruction,  and  that  it  has  to  be  converted 
and  reorganized  into  procedures  through  experience.  IKIS  combines 
Socratic  style  tutoring  and  coaching  with  simulation  to  help 
strengthen  and  exercise  cognitive  components  of  the  subject  domain 
and  enhance  the  learners7  awareness  of  their  own  abilities  and 
confidence  in  the  use  of  the  field  unit.  Specifically  IKIS 
cognitive  objectives  include: 

*  Conceptualize  relationships  that  exist  between 
transponder  faults,  test  set  faults,  and  user  induced 
faults . 

*  Integrate  factual  components  related  to  test  set 
positioning  and  aiming  with  corresponding  display 
readouts  and  and  system  faults. 

*  Internalize  procedural  methods  required  for  Test  Set 
verification  and  operation  and  there  relationships  with 
Test  Set  display  characteristics. 


Adding  the  knowledge  of  an  expert  teacher  to  provide  an  adaptive 
tutoring  system  requires  the  design  and  implementation  of  an 
overall  teaching  strategy.  Creating  this  level  of  knowledge 
(metaknowledge)  was  not  a  goal  of  the  Phase  I  research.  However 
since  this  is  a  goal  of  the  IKIS  project,  the  implementation  had  to 
be  considered  in  the  system  design.  The  major  components  of  the 
IKIS  design  strategy  are  described  below. 


MEASURING  BASELINE  PERFORMANCE 


Measuring  a  learner's  entering  domain  related  performance  skills  is 
necessary  for  the  tutor  to  adapt  to  the  individuals  needs.  Once 
the  cognitive  and  behavioral  objectives  have  been  defined,  the 
relative  strength  of  responses  related  to  those  objective  can  be 
measured  under  appropriate  stimulus  conditions.  Pretesting  the 
learner  with  questions  related  to  the  learning  objectives  is  a 
useful,  but  not  necessarily  sufficient,  method  of  measuring  this 
baseline. 

Historically,  many  student  who  perform  well  on  written  exams  still 
cannot  perform  well  on  related  tasks.  This  is  not  surprising, 
since  the  stimuli  that  evoke  the  responses  are,  in  fact,  very 
different.  The  learner  has  had  little  chance,  or  help,  in 
resorting  the  cognitive  components  of  the  task  being  taught.  High 
fidelity  simulation  provides  a  stimulus  situation  more  closely 
related  to  the  stimuli  which  ultimately  will  evoke  appropriate 
responses. 

The  simulation  implemented  in  the  Phase  I  prototype  provides  an 
environment  for  measuring  baseline  behaviors  under  stimulus 
conditions  more  closely  related  to  those  of  the  operational 
environment.  IKIS  can  simulate  complex  situations  that  require  the 
discrimination  of  a  variety  of  stimuli  and  the  integration  of 
system  functional  relationships  for  the  operator  to  respond 
appropriately.  Fault  identification,  for  example,  requires 
understanding  system  operation  as  well  as  integration  of 
relationships  between  positioning  and  aiming  the  test  set,  loading 
of  system  codes,  and  viewfinder  display  codes.  During  Phase  II 
behavioral  measurements  of  operator  performance  related  to  a 
complex  task  as  well  as  each  component  of  the  task  will  be  made. 
Such  measurements  reflect  both  the  cognitive  and  behavioral 
performance  of  the  individual  learner. 


The  system  reaches  its  behavioral  objectives  by  reinforcing 
successive  approximations  toward  the  desired  behaviors.  A  stimulus 
is  presented  that  provides  an  occasion  upon  which  the  learner  is 
likely  to  respond  appropriately.  Responses  are  then  compared  to 
baseline  measurements.  Those  that  are  successively  closer  to  the 
desired  behaviors  are  strengthened  by  reinforcement.  Initially 
stimuli  presented  to  the  learner  should  be  designed  so  that  the 
learner  can  easily  discriminate  it  from  the  background  noise.  Once 
the  desired  response  has  been  sufficiently  strengthened,  the 
discriminative  stimulus  is  gradually  faded  until  it  takes  on  those 
properties  that  approximate  real  world  stimuli  as  closely  as 
possible . 

IKIS  is  designed  to  provide  appropriate  stimuli  through  the 
Socratic  tutor,  coach,  and  simulator.  In  the  Phase  I  prototype  the 
user  is  directed,  in  small  steps,  to  perform  each  task  related  to 
Test  Set  operation.  Help  is  available  from  the  tutor,  coaching, 
and  simulation  to  help  identify  system  components  and  clarify 
procedures.  Natural  reinforcers  associated  with  successful 
operation  and  correct  functioning  of  the  the  Test  Set  are  available 
through  the  simulation. 

As  metaknowledge  is  added  to  the  system  during  Phase  II,  stimuli 
appropriate  for  learning  higher  order  knowledge  in  small  steps  will 
be  added  to  the  system.  Stimuli  that  the  learner  should 
discriminate  will  be  displayed  initially  at  full  intensity  while 
non  related  stimuli  can  be  diminished  in  intensity.  For  example  a 
fault  due  to  incorrectly  aiming  the  Test  Set  will  result  in  faulty 
data  being  displayed  by  one  or  more  of  the  LEDs'.  By  diminishing 
all  display  components  except  for  the  aim  and  the  appropriate 
LEDs',  the  user  will  more  easily  discriminate  those  stimuli  from 
the  background.  Coaching  will  help  provide  additional  supportive 
semantic  knowledge  related  to  the  functional  relationships 
producing  the  faults.  As  the  users'  accuracy  at  identifying  and 
correcting  this  fault  increases,  the  differences  in  stimulus 
intensity  from  the  background  will  be  decreased.  In  this  v/ay  the 
student  will  gradually  learn  to  make  the  correct  discriminations 
related  to  identification  of  that  fault. 


IMMEDIATE  FEEDBACK 


Immediate  feedback  following  learner  responses  has  been  shown  to  be 
an  important  and  powerful  teaching  tool.  Choosing  the  appropriate 


moment  and  form  of  feedback  is  critical  to  the  learning  process. 
Feedback  that  appears,  to  the  system  designer,  to  be  reinforcing 
may  prove  to  be  neutral  or  even  punishing  to  some  learners. 
Reinforcers  are  determined  by  their  effects  on  the  learner,  not  the 
teacher.  The  quality  and  quantity  of  feedback  useful  for 
maximizing  the  learning  process  will  very  from  individual  to 
individual  and  lesson  to  lesson.  This  is  another  area  in  which 
the  system  must  adapt  to  the  learner. 

Tasks  that  have  complex  behavioral  and  cognitive  components  often 
occur  in  sequences.  Under  such  conditions  responses  at  one  point 
of  the  sequence  serve  as  stimuli  for  the  next  chunk  of  cognitive 
and  behavioral  events.  Immediate  feedback  provided  for  a  response 
in  the  sequence  can  disrupt  the  sequence,  be  punishing  to  the 
learner,  and  detrimental  to  the  overall  learning  process.  Small 
errors  on  the  part  of  the  learner  may  not  be  significant  enough  to 
warrant  stopping  the  simulation  to  provide  immediate  feedback.  In 
such  cases  it  is  better  to  provide  a  less  intrusive  means  for  the 
learner  to  discriminate  self  errors.  Icons  and/or  symbols  can  be 
displayed  when  and  where  the  error  is  detected  to  flag  the  learner 
of  an  incorrect  response. 

The  IKIS  Phase  I  project  provides  feedback  primarily  through  the 
coach.  As  mentioned  above,  passive  feedback  is  available  to  the 
user  for  discovering  functions  of  the  viewfinder  components 
including  the  LED's  and  "display  lamps.  Active  feedback  is 
demonstrated  when  an  error  in  disconnection  of  the  KIR-l  computer 
is  made  by  the  learner.  This  is  a  sever  enough  error  that  the 
student  is  briefly  interrupted  and  told  of  the  error.  The  student 
is,  however,  free  to  continue  with  the  simulation  once  they  nave 
acknowledged  the  error. 


As  metaknowledge 
adaptive  feedback 
calls  for  using 
effectiveness  of 
statistics  of  all 
more  adaptive  to 
problems . 


is  added  to  the  system  during  Phase  II  a  more 
mechanism  will  be  implemented.  The  system  design 
behavioral  measurements  as  an  indicator  of  the 
the  feedback  provided.  In  addition,  as  global 
users  are  collected  the  system  will  learn  to  be 
specific  types  of  learners  and  specific  types  of 
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SYSTEM  ARCHITECTURE 


IKIS  is  composed  of  five  major  components;  the  user  interface, 
tutor,  simulator,  student  model,  and  inference  engine.  The  diagram 
shown  in  figure  6,  below,  illustrates  how  these  components  are 
related. 


/t'~.  USER  mb 

interface  gigs 

'tfSf i  'jS.-  _  jHpff 

*  t.  r*-x*  --  '  \ 

i V'An  ‘ , v  -k  - ,>v 


SIMULATION  L 


\y<-  ty  «  *.  *  v  *».• 

^ #  v  */«-'•• ..  ,  iV:  — . 

\  1* \  v>;‘ iX-.-T 


a> 

STUDENT 

MODEL 


EXPERT  I 
SYSTEM  | 


i 


tutor] 


Figure  6.  System  Architecture 


The  prototype  software  was  developed  using  the  Smalltalk  V 
language.  Smalltalk  is  an  object  oriented  language  that  supports 
mouse  input  and  high  resolution  color  graphics  compatible  with  an 
IBM  Enhanced  Graphics  Adapter. 

The  IKIS  Phase  I  project  produced  almost  1  megabyte  of  high 
resolution  color  graphical  images  and  100  Kilobytes  of  Smalltalk 
source  code.  The  Smalltalk  image,  consisting  of  all  Smalltalk 
classes  plus  IKIS  object  source  code  is  approximately  1.2  Megabytes 
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in  size.  The  frequent  swapping  out  of  Smalltalk  objects  to  disk 
decreases  the  processing  speed  of  IKIS.  Currently  a  version  of 
Smalltalk  that  utilizes  the  expanded  memory  of  the  IBM  AT  is  being 
beta  tested  by  this  company.  When  complete  this  version  should 
execute  all  IKIS  code  without  the  need  to  swap  out  objects, 
dramatically  increase  the  speed  at  which  IKIS  operates. 


USER  INTERFACE 


IKIS  employs  a  high  fidelity  color  graphics  interface.  Functional 
components  of  the  program  run  independently  in  separate  windows  on 
the  display.  A  mouse  pointing  device  handles  almost  all  input  from 
the  user.  User  input  and  system  navigation  occurs  through  the 
selection  of  options  displayed  in  pop-up  menus.  Each  window  has 
two  or  more  pop-up  menus.  One  is  associated  with  the  top  I  bar  of 
the  window.  The  second  is  associated  with  the  window  pane  itself. 

A  menu  is  activated  by  pointing  to  the  position  described  above  and 
pressing  the  right  mouse  button.  Once  the  menu  is  active,  choices 
on  the  menu  are  highlighted  by  moving  the  mouse  forward  or  backward 
in  the  menu.  Once  a  choice  is  highlighted,  it  can  be  activated  by 
pressing  the  right  mouse  button.  An  inactive  window  can  be 
activated  by  selecting  the  "cycle”  option  in  the  I  bar  menu  or  by 
simply  pointing  in  one  of  the  background  windows  and  pressing  the 
left  mouse  button.  Selecting  "help"  from  a  pop-up  menu  will 
activate  the  tutor  window  and  display  a  context  sensitive  help 
screen. 

TUTOR 


IKIS  provides  tutoring  capabilities  through  a  Socratic  Tutor  and  a 
Coach.  In  addition  the  simulator  and  coaching  mechanisms  provide 
learners  with  a  discovery  mechanism  through  which  they  can  by 
exploring  system  components,  functions,  and  relationships.  Context 
sensitive  help  is  available  from  most  system  menus. 


Socratic 


The  tutor  is  evoked  from  the  top  menu  of  the  viewfinder  or  by 
mousing  on  the  tutor  window.  In  addition  special  tutoring  screens 
are  activated  when  the  user  requests  help  from  other  system 
windows . 

The  Socratic  Tutor  provides  zeroth  order  textbook  knowledge  to  the 
student  in  a  separate  window  on  the  display.  Designed  as  a  finite 
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state  system,  the  tutor  displays  separate  instances  of  the  state 
object.  This  design  creates  handy  mechanisms  for  adding  and 
updating  semantic  components  and  their  links  to  other  learning 
states  and  releases  the  code  from  order  dependence. 

Each  tutoring  state  contains  transition  rules  that  determine  what 
the  next  state  will  be  based  on  user  input.  The  state  system  tutor 
design  forms  a  core  element  in  the  development  of  an  expert  tutor 
authoring  system.  In  later  phases,  the  designers  can  implement  an 
editor  for  easily  creating  states  and  their  links  and  reducing  the 
cost  and  complexity  of  authoring  new  lessons. 


Coaching 

The  coaching  mechanism  provides  an  additional  means  of  tutoring. 
The  system  supports  both  passive  and  active  coaching.  Passive 
coaching  is  used  for  identification  of  components  of  the  Test  Set. 
Using  the  mouse  to  point  the  cursor  at  a  lamp,  LED,  or  other 
component  of  the  viewfinder  results  in  a  description  of  the  device 
being  displayed. 

Active  coaching  begins  the  moment  the  learner  incorrectly  performs 
a  procedure.  An  example  of  this  has  been  implemented  in  the  Phase 
I  prototype.  If  the  KIR-1  computer  is  not  disconnected  in  the 
correct  order  loss  of  the  mode  4  code  can  occur.  IKIS  interrupts 
the  student  when  this  error  is  detected  and  coaches  the  student, 
describing  both  the  error  and  correct  procedure. 

Additional  active  and  passive  coaching  will  be  provided  in  Phase  II 
as  higher  levels  of  knowledge  are  added  to  the  knowledge  base. 
This  is  one  way  in  which  the  system  can  become  more  adaptive  to 
individual  learners. 


Discovery 


Discovery  learning  results  from  using  experimentation  with  the 
components  of  the  simulator  and  requests  for  deeper  levels  of 
knowledge  about  component  and  functional  relationships.  Passive 
coaching,  a  component  of  discovery  learning,  provides  requested 
information  to  the  learner  about  the  components  being  experimented 
with.  In  Phase  II,  deeper  levels  of  knowledge  will  be  added  to  the 
coaching  mechanism  to  provide  greater  breadth  to  this  component. 


SIMULATION 
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The  simulation  of  Test  Set  functional  operation  is  a  major 
component  of  IKIS.  From  the  Test  Set  window  the  learner  can 
simulate  the  procedural  components  related  to  connecting  the  KIR-l 
computer  and  cables  to  the  Test  Set  and  power  supply,  loading  mode 
4  into  the  KIR-1  computer  and  disconnecting  the  KIR-1  and  cables. 
These  concepts  and  procedures  can  then  be  cognitively  assembled  by 
the  learner. 

The  component  parts  of  the  simulation  include  the  viewfinder 
window,  LED',s  test  lamps,  Test  Set,  switches,  KIR-1  computer, 
cables,  test  zones,  and  the  helicopters.  All  are  represented  as 
objects  in  the  system.  The  current  state  of  the  simulation  is 
changed  and  updated  by  messages  sent  to  these  objects  from  the 
inference  engine.  The  knowledge  base  contains  facts  related  to 
current  inputs  affecting  the  state  of  the  simulation.  A  change  in 
the  facts  related  to  the  simulation  results  in  a  change  in  the 
current  simulation  state. 

The  viewfinder  window  is  a  major  component  of  the  simulation. 
Looking  through  the  viewfinder  the  learner  can  simulate  aiming  the 
test  set  at  the  aircraft  IFF  antenna.  The  viewfinder  sight  must  be 
lined  up  within  5°  of  the  aircraft's  IFF  antenna  for  correct 
operation.  The  learner  aims  at  the  antenna  by  pointing  and 
clicking  the  mouse  in  the  “directional  arrows  to  the  left  of  the 
sight.  A  simulation  of  the  viewfinder's  LEDs  and  test  lamps  is 
also  provided.  The  inference  engine  sends  messages  to  update  these 
LEDs  and  lamps.  The  simulation  displays  readouts  similar  to  those 
that  would  be  obtained  in  actual  use  including  erroneous  data  if 
the  Test  Set  is  operated  incorrectly. 

Preferred  test  zones  are  shown  in  the  test  zone  window.  The 
learner  positions  himself  anywhere  on  the  test  zone  map  by  pointing 
and  clicking  with  the  mouse.  His  position  is  indicated  by  a  red 
dot  displayed  at  that  position.  The  helicopter,  shown  through  the 
display,  will  change  its  position  relative  to  the  position  the 
operator  has  chosen  in  the  test  zone.  Whether  or  not  the  operator 
is  correctly  positioned  in  a  preferred  test  zone  is  a  fact  sent  to 
the  fact  base  and  used  by  the  inference  engine  to  control 
functioning  of  the  test  set  objects. 

Test  Set  switches  on  the  handle  of  the  Test  Set  are  shown  through  a 
porthole  in  the  upper  left  of  the  viewfinder  window  as  well  as  in 
the  outside  view  of  the  Test  Set.  Pointing  and  clicking  with  the 
mouse  simulates  pressing  the  switches.  The  switch  "pressed"  is 
indicated  as  a  fact  added  to  the  fact  base.  The  inference  engine, 
upon  scanning  the  fact  base,  updates  the  state  of  the  simulation. 
The  fact  is  then  removed  from  the  fact  base  until  the  switch  is 
pressed  again. 
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The  outside  view  of  the  Test  Set  is  shown  in  a  separate  window. 
Selections  shown  on  a  pop-up  menu  allow  the  learner  to  simulate 
connecting  (or  disconnecting)  the  Test  Set  to  the  KIR-1  computer, 
connecting  cables,  and  to  a  power  source.  The  menu  also  provides 
for  simulating  the  loading  of  mode  4  codes  into  the  KIR-1  computer. 
Facts  are  added  or  removed  from  the  fact  base  related  to  the  state 
of  each  of  these  components. 

Use  of  the  simulator  provides  the  learner  with  the  chance  to 
integrate  both  behavioral  and  cognitive,  procedural  and  factual 
components  of  test  set  operation.  This  provides  a  deeper  level  of 
learning  then  is  possible  through  reading  the  instructional  manual 
alone. 


STUDENT  MODEL 


The  student  model  is  a  separate  object  in  the  system  that  stores 
facts  about  each  individual  student.  When  a  new  student  first  uses 
the  system  it  creates  a  file  containing  an  instance  of  this  object. 
As  the  student  interacts  with  the  system,  the  student  model 
maintains  information  about  the  current  state  of  the  simulation. 
The  students  file  is  updated  by  storing  the  updated  instance  on 
disk  as  the  student  exits  the  program.  Upon  reentry,  the  student 
returns  to  the  state  of  the  simulation  when  last  used. 

Each  instance  of  the  student  model  is  initialized  with  a  list  of 
possible  faults  that  may  result  through  an  error  in  the 
transponder,  the  Test  Set,  or  due  to  operator  error.  The  fault 
list  is  currently  arranged  in  an  increasing  order  of  difficulty  of 
diagnosis.  When  the  fault  has  been  inserted,  the  student  may 
identify  the  fault  and  then  indicate  the  appropriate  corrective 
action.  Then  the  fault  is  removed  from  the  student's  fault  list. 
If  a  fault  is  not  correctly  identified  the  student  may  still  remove 
it.  However  then  the  fault  is  returned  to  the  students  fault  list. 

Further  development  of  the  student  model  during  Phase  II  will 
increase  the  ability  of  the  system  to  adapt  to  the  individual 
student.  This  is  an  area  of  exciting  cognitive  and  behavioral 
research.  Performance  measurements  related  to  accuracy,  response 
latency,  and  frequency  collectively  reflect  the  learners  cognitive 
processing  of  information.  The  IKIS  Phase  I  project  provides  an 
environment  for  empirical  testing  of  relationships  between  system 
motivational  components,  student  performance,  and  cognition. 
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INFERENCE  ENGINE 


The  heuristical  knowledge  of  both  domain  experts  and  an  expert 
teacher  is  stored  in  the  system  inference  engine.  The  engine 
operates  on  facts  provided  to  it  by  other  objects  including  the 
simulator,  tutor,  and  student  model.  The  fact  base  is  designed  as 
a  global  dictionary  to  which  facts  can  be  added  or  removed,  based 
on  system  activity  and  user  interaction.  The  inference  engine 
implemented  in  Phase  I  is  forward  chaining.  Each  rule  in  the 
inference  engine  consists  of  one  or  more  facts  which,  if  true, 
cause  some  action  to  occur.  One  possible  action  that  may  occur 
when  a  rule  is  fired  is  the  production  of  a  new  fact.  Therefore 
when  the  inference  engine  is  evoked  the  rule  base  is  scanned 
repeatedly  until  no  rules  are  left  to  fire. 

Continued  research  and  development  during  Phase  II  will  focus  on 
adding  to  and  refining  the  inference  engine  and  rule  base  to 
provide  higher  orders  of  knowledge,  including  metaknowledge,  and 
more  sophisticated  reasoning.  Interactions  between  the  inference 
engine  and  facts  provided  by  other  system  objects,  especially  the 
student  model,  will  be  the  key  to  full  development  of  an  adaptive 


CONCLUSIONS 


Phase  I  of  the  IKIS  project  has  produced  the  elements  of  a 
sophisticated  simulation-based  intelligent  tutoring  system.  Its 
student  model,  teaching  strategies,  and  discovery  system  can  be 
extended  into  a  fieldable  and  cost  effective  tool. 

Moreover,  the  modular  design  of  IKIS  makes  such  extension  possible. 
Each  system  component  can  be  modified  and  revised  independent  of 
other  modules  in  the  system.  States  can  be  augmented,  modified,  or 
deleted  from  the  tutor  with  ease.  Tools  to  measure  and  record 

learner  behaviors  easily  can  be  added  to  the  student  model. 
Additional  facts  can  be  incorporated  readily  into  the  fact 

dictionary  from  the  student  model  and  simulation.  Even  components 
of  the  simulation  can  simply  be  modified  and  changed  as  needed. 

In  addition  the  results  of  the  Phase  I  project  suggest  approaches 
and  tools  for  research  into  relationships  between  learning, 
cognitive  load,  and  performance.  The  subject  domain  of  the  Phase  I 
research  did  not  place  a  severe  enough  cognitive  load  on  the 

learner.  Hence  it  may  be  necessary  to  apply  IKIS  to  a  more 

sophisticated  training  problem  to  provide  sufficient  cognitive  and 
behavioral  requirements  for 'meaningful  future  research. 

Simulation  based  intelligent  tutoring  systems  ultimately  can 

provide  more  efficient  and  effective  training  of  complex  tasks 
than  conventional  methods.  Furthermore  it  can  provide  a  more 

objective  analysis  of  individual  user  aptitude  and  performance  than 
has  been  possible  with  other  techniques.  The  IKIS  Phase  I  project 

provides  a  test  bed  for  further  research  and  development  of 

simulation  based  training  for  ever  more  sophisticated  military  and 
industrial  equipment. 


RECOMMENDATIONS 


The  training  and  measurement  of  cognitive  skills  related  to  real 
time  decision-making  is  an  area  in  which  simulation  based 
intelligent  tutoring  systems  like  IKIS  can  have  a  significant 
effect.  Other  research  by  MICROEXPERT  suggests  that  in  similar 
tasks  such  as  intelligence  message  routing  and  missile  radar 
training,  teaching  of  semantic  knowledge  must  be  supplemented  by 
syntactic  approaches.  "Doing"  in  such  tasks  becomes  the  measure  of 
"knowing. " 

The  personnel  who  currently  teach  and  grade  this  type  of 
performance  training  cannot  always  provide  a  consistent  quality  of 
training  or  sufficient  objective  analysis  of  learner  performance. 
Research  during  Phase  II  should  focus  on  teaching  real-time 
cognitive  skills  through  measurable  results  and  student  performance 
analysis  for  increasingly  complex  tasks. 

Hence,  research  and  development  during  Phase  II  should  concentrate 
on  measuring  and  training  individuals  in  tasks  having  complex 
cognitive  and  behavioral  performance  components.  In  particular  the 
Phase  II  project  should  focus  on  the  following: 

1.  RESEARCH  ON  TEACHING  STRATEGIES  AND  METHODS.  Communication  to 
the  learner  is  based  on  using  interactive  simulation  to 
encourage  a  semantic  rationalization  process.  The  simulation 
provides  an  interactive  diagnostic  tool  for  constant 
evaluation  of  the  learner.  The  relationships  between 
observable  and  quantifiable  behavioral  sequences  and 
cognition  should  be  studied  further.  This  data  can  be  used 
for  developing  didactic  operations  and  plans  of  action  by  the 
system. 

2.  IMPLEMENTATION  OF  DEEPER  KNOWLEDGE.  Data  gained  from  research 
described  above  will  result  in  additional  types  of  knowledge, 
especially  metaknowledge,  that  can  be  added  to  the  inference 
engine.  This  knowledge  will  be  useful  in  allowing  the  system 
to  adapt  to  individual  learners  and  maximize  learner 
performance . 

3.  AUGMENTATION  OF  SYSTEM  INTELLIGENCE.  Features  to  be  added  to 
the  inference  engine  include  backward  chaining,  use  of  global 
variables,  adding  conflict  resolution  (tie-breaking) 
procedures.  Other  enhancements  will  also  be  reviewed  for 
possible  adoption. 
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4.  ADDITION  OF  RECORD-KEEPING  MODULE.  Student  performance 
records  should  be  available  for  viewing  by  superiors, 
trainers,  and  the  learner  as  a  means  of  system  refinement  and 
as  a  tool  for  evaluating  the  learner. 

5.  ENHANCEMENT  OF  KNOWLEDGE  REPRESENTATION.  Techniques  to 
organize  knowledge  into  data  structures  for  system 
manipulation  should  be  further  developed.  Task  analysis  is 
an  important  step  not  always  utilized  in  the  developing  of  a 
knowledge  base.  Building  on  the  task  analysis  completed 
during  Phase  I,  additional  diagnostic  rules  and  causal 
relationships  should  be  added  to  the  knowledge  base.  The 
knowledge  base  can  then  be  used  to  generate  an  "expert's 
solution,"  which  when  compared  to  the  student  behavior, 
provides  a  basis  for  advising  the  student. 

The  MICROEXPERT  design  team  has  attempted  to  cut  development  time 
and  the  cost  to  bring  up  a  new  tutor  by  attempting  to  keep  the 
components  of  the  system  modular.  By  maintaining  modularity,  it 
should  be  possible  to  create  new  tutors  for  related  domains  of 
training  with  reduced  effort.  While  changes  and  additions  to 
system  components  are  required,  tutoring  control  and  the  basic 
components  of  the  student  model  and  inference  engine  can  be 
retained  and  built  upon. 


